Additional outputs to longitudinal functional workflow Fix #190 by jpillai00 · Pull Request #282 · childmindresearch/rbc

jpillai00 · 2026-04-08T23:43:11Z

The longitudinal functional workflow previously only warped cross-sectional outputs (sbref, preproc BOLD, brain mask, and the BOLD-to-longitudinal transform) into longitudinal template space. This PR extends it to recompute nuisance regression in longitudinal space using the raw (non-bandpass-filtered) regressors from cross-sectional preprocessing.

Key changes:

FunctionalLongOutputs now includes regressed_bold and cleaned_bold as dict[str, Path], supporting multiple regressor strategies
longitudinal_process accepts regressor_files: dict[str, Path] and re-runs both non-bandpassed and bandpassed regression in longitudinal space
FunctionalOutputs separately exports regressor_file (raw, passed to longitudinal workflow) and bpf_regressor_file (bandpass-filtered) - in cross sectional
resolve_longitudinal_func resolves raw regressor files per regressor strategy
export_longitudinal_func saves per-regressor regressed_bold and cleaned_bold
--regressor argument added to the longitudinal CLI

github-actions · 2026-04-08T23:43:53Z

Coverage Report

File	Stmts	Miss	Cover	Missing
rbc
__init__.py	1	0	100%
context.py	25	8	68%	70, 75–77, 79–80, 93–94
metadata.py	56	0	100%
rbc/bids
__init__.py	9	0	100%
_schema.py	585	4	99%	776, 782, 790, 1101
anatomical.py	24	3	87%	44, 47–48
builder.py	72	7	90%	233–235, 362, 364–365, 386
functional.py	43	5	88%	46–49, 84
longitudinal.py	36	0	100%
metrics.py	23	1	95%	44
qc.py	17	0	100%
query.py	67	42	37%	103–107, 121–125, 127–128, 130–135, 137, 139, 153, 159–165, 198, 207, 209–216, 225, 257, 266–267
session.py	47	0	100%
rbc/cli
__init__.py	1	0	100%
all.py	49	2	95%	84, 109
anatomical.py	25	2	92%	47, 63
base.py	71	9	87%	56, 62, 123–125, 131–133, 156
functional.py	33	2	93%	67, 87
longitudinal.py	29	2	93%	48, 66
main.py	42	0	100%
metrics.py	42	2	95%	77, 95
qc.py	25	2	92%	45, 61
rbc/core
__init__.py	3	0	100%
common.py	26	12	53%	43–45, 62–70
fileops.py	27	4	85%	69–72
fsl2itk.py	42	0	100%
nifti.py	192	5	97%	236–237, 244–245, 524
niwrap.py	56	1	98%	58
rbc/core/anatomical
__init__.py	4	0	100%
registration.py	15	4	73%	59, 151, 166, 183
segmentation.py	24	8	66%	64, 74–76, 92, 114, 125, 141
rbc/core/functional
__init__.py	13	0	100%
coregistration.py	7	2	71%	44, 55
despiking.py	7	3	57%	32, 36–37
distortion.py	130	40	69%	269–271, 321, 324, 332–335, 341–346, 349, 352–353, 356, 365–369, 375–376, 387–388, 391, 397, 443–444, 447, 455, 461, 470–471, 474, 484, 491
erosion.py	32	1	96%	50
initialization.py	9	4	55%	35, 42–43, 63
masking.py	34	25	26%	53, 55–56, 58–59, 62–65, 69, 91, 134, 183, 197, 208, 223, 233, 249, 258, 271, 285, 296, 306, 319, 328
motion.py	57	37	35%	62, 64–67, 69, 71, 73, 76–77, 83–84, 86–87, 95–97, 99, 102, 105, 107, 124–125, 135–138, 159, 169, 171–172, 175, 177–179, 181, 183
nuisance.py	81	60	25%	78, 80–85, 87, 89–90, 93, 96–98, 100–102, 104–110, 112–116, 118, 163, 165, 167, 170–171, 173–175, 178, 181–182, 185–187, 190–193, 197, 203–205, 207, 235, 243, 269, 278, 307, 316, 322
regressors.py	89	6	93%	163, 193, 325–328
resampling.py	54	43	20%	37–42, 74–76, 78, 80–81, 85–87, 90, 93, 105, 107–108, 112, 114, 150–152, 154–155, 159, 161–162, 167–169, 173–174, 177, 180, 187, 199, 201–202, 207, 209
timing.py	16	11	31%	46–47, 49–53, 58, 60, 66–67
rbc/core/longitudinal
__init__.py	1	0	100%
transform.py	46	7	84%	106–107, 165–168, 170
rbc/core/metrics
__init__.py	3	0	100%
alff.py	90	1	98%	265
reho.py	66	0	100%
smoothing.py	7	3	57%	36, 42–43
standardization.py	27	11	59%	64, 66–68, 70, 72–75, 77–78
timeseries.py	57	1	98%	120
rbc/core/qc
__init__.py	6	0	100%
dvars.py	26	0	100%
motion.py	31	0	100%
registration.py	41	0	100%
xcp.py	41	0	100%
rbc/orchestration
__init__.py	32	2	93%	90, 93
all.py	41	9	78%	131–132, 137, 145, 153–154, 171, 173–174
anatomical.py	37	22	40%	49–52, 57–58, 60–63, 85–87, 89–90, 94, 101, 104, 109–110, 116, 118
functional.py	41	12	70%	124–126, 128–129, 133, 139, 142, 147–148, 157, 159
longitudinal.py	56	1	98%	156
metrics.py	46	30	34%	32–35, 38–40, 67, 75–76, 100–102, 104–106, 110, 118–121, 123–124, 130–132, 137, 145, 152, 154
qc.py	34	0	100%
rbc/workflows
__init__.py	10	0	100%
anatomical.py	47	19	59%	90–93, 97–102, 107, 177–180, 182–184, 188
functional.py	111	66	40%	115–116, 124, 129–130, 138, 209–210, 213–214, 217–218, 221, 224–227, 237–239, 249–250, 253, 256, 259–260, 268–269, 276–277, 284–285, 292–293, 296–298, 301–304, 316–317, 329, 331–334, 337–338, 347–348, 356, 363, 455, 462, 465, 470, 472–474, 476–477, 483–484, 490
metrics.py	41	18	56%	80, 83–84, 93–94, 99–102, 105–108, 112–115, 119
qc.py	55	34	38%	88–89, 92–93, 96, 99, 102–107, 110, 119–121, 125–128, 130–134, 136–137, 139–141, 144, 161, 167, 169
rbc_resources
__init__.py	38	0	100%
TOTAL	3271	593	81%

Tests	Skipped	Failures	Errors	Time
729	0 💤	0 ❌	0 🔥	10.132s ⏱️

Copilot

Pull request overview

Extends the longitudinal functional workflow to re-run nuisance regression in longitudinal template space using the raw (non-bandpass-filtered) nuisance regressors produced during cross-sectional preprocessing, while also exporting additional regressor artifacts from the cross-sectional stage.

Changes:

Expand functional output structures to carry both raw and bandpass-filtered regressor .1D files, and add longitudinal-space regressed_bold/cleaned_bold outputs per regressor strategy.
Update longitudinal orchestration/BIDS resolve+export to accept a regressor strategy list and to resolve raw regressor files per strategy.
Add --regressor support to the longitudinal CLI and update unit tests accordingly.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/orchestration/test_longitudinal.py	Updates mocks and `process_func` calls to pass regressors; adjusts/removed tests impacted by new required inputs.
tests/unit/cli/test_longitudinal.py	Adds `regressor` to the longitudinal CLI test namespace defaults.
tests/unit/bids/test_exports.py	Updates expected derivative file counts due to new exported regressor artifact(s).
src/rbc/workflows/functional.py	Adds raw+filtered regressor outputs to cross-sectional outputs; adds longitudinal-space regression outputs and implements re-regression in longitudinal space.
src/rbc/orchestration/longitudinal.py	Threads regressor strategies through longitudinal functional processing; adds default regressors parameter.
src/rbc/cli/longitudinal.py	Adds `--regressor` CLI option and passes it through to orchestration.
src/rbc/bids/longitudinal.py	Resolves raw regressor `.1D` files per strategy and exports longitudinal-space regressed/cleaned BOLD outputs.
src/rbc/bids/functional.py	Exports an additional bandpass-filtered regressor `.1D` artifact per regressor strategy.

Comments suppressed due to low confidence (1)

tests/unit/orchestration/test_longitudinal.py:346

test_missing_required_file_raises no longer covers the newly-required longitudinal inputs (bold_mask and the per-regressor regressor_files). Since resolve_longitudinal_func() now uses expect() for these, it would be good to extend the parametrization to assert a FileNotFoundError when the mask or a regressor .1D file is missing (per regressor strategy).

    @pytest.mark.parametrize(
        ("match_field", "match_kwargs"),
        [
            ("bold", {"suffix": "bold", "desc": "preproc"}),
            ("sbref", {"suffix": "sbref"}),
            (
                "bold_to_anat_xfm",
                {
                    "suffix": "xfm",
                    "desc": "linear",
                    "extension": ".txt",
                    "extra": {"from": "bold", "to": "T1w", "mode": "image"},
                },
            ),
        ],

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kaitj

Largely looks good to me, a couple of minor changes with the type hints and casting. Also looks like it needs a rebase now.

One thing that comes to mind since the longitudinal currently relies on flags to denote the modality to run (--anatomical, --functional), should there be an early check to ensure the anatomical has been run? It should error out pretty early none the less since the data is relied upon on in the first step.

nx10 · 2026-04-13T17:45:12Z

Thanks @jpillai00 — the regressor_file vs bpf_regressor_file split and per-strategy regressed_bold/cleaned_bold dicts are the right shape; they just need to land on the new submodule layout. Closing as superseded by #301; Stage 5 rewrites this against workflows/longitudinal/functional.py. Port commits will carry Co-authored-by: Janhavi Pillai trailers.

Closed as part of #301 Stage 0.

regressor_file now carries the raw (unfiltered) regressor .1D files as computed from native-space BOLD. The new bpf_regressor_file field holds the bandpass-filtered version that 3dTproject actually applied, exported with desc-<strategy>Filtered for provenance. This lets the longitudinal pipeline reuse the raw regressors without recomputation, matching the cross-sectional principle: one regressor computation per run, applied in each target space. Port of the regressor-split idea from #282. Co-authored-by: Janhavi Pillai <janhaviiyengar@gmail.com>

FunctionalLongOutputs gains regressed_bold and cleaned_bold dicts keyed by regressor strategy. longitudinal_process now accepts regressor_files (raw .1D from cross-sectional) and re-runs apply_regression + apply_regression_bandpass on the warped BOLD. No regressor recomputation -- same matrix, different target space. bold_mask is now mandatory (was Path | None). BIDS export writes per-regressor desc-regressed and desc-preproc BOLD with reg-<strategy> entity. resolve_longitudinal_func resolves the raw regressor files per strategy. CLI gains --regressor with the same choices as cross-sectional (36-parameter, aCompCor). Port of the longitudinal regression logic from #282. Co-authored-by: Janhavi Pillai <janhaviiyengar@gmail.com>

regressor_file now carries the raw (unfiltered) regressor .1D files as computed from native-space BOLD. The new bpf_regressor_file field holds the bandpass-filtered version that 3dTproject actually applied, exported with desc-<strategy>Filtered for provenance. This lets the longitudinal pipeline reuse the raw regressors without recomputation, matching the cross-sectional principle: one regressor computation per run, applied in each target space. Port of the regressor-split idea from #282. Co-authored-by: Janhavi Pillai <janhavi.pillai@gmail.com>

FunctionalLongOutputs gains regressed_bold and cleaned_bold dicts keyed by regressor strategy. longitudinal_process now accepts regressor_files (raw .1D from cross-sectional) and re-runs apply_regression + apply_regression_bandpass on the warped BOLD. No regressor recomputation -- same matrix, different target space. bold_mask is now mandatory (was Path | None). BIDS export writes per-regressor desc-regressed and desc-preproc BOLD with reg-<strategy> entity. resolve_longitudinal_func resolves the raw regressor files per strategy. CLI gains --regressor with the same choices as cross-sectional (36-parameter, aCompCor). Port of the longitudinal regression logic from #282. Co-authored-by: Janhavi Pillai <janhavi.pillai@gmail.com>

* Split FunctionalOutputs.regressor_file into raw + bandpass-filtered regressor_file now carries the raw (unfiltered) regressor .1D files as computed from native-space BOLD. The new bpf_regressor_file field holds the bandpass-filtered version that 3dTproject actually applied, exported with desc-<strategy>Filtered for provenance. This lets the longitudinal pipeline reuse the raw regressors without recomputation, matching the cross-sectional principle: one regressor computation per run, applied in each target space. Port of the regressor-split idea from #282. Co-authored-by: Janhavi Pillai <janhavi.pillai@gmail.com> * Add per-regressor regression to longitudinal functional workflow FunctionalLongOutputs gains regressed_bold and cleaned_bold dicts keyed by regressor strategy. longitudinal_process now accepts regressor_files (raw .1D from cross-sectional) and re-runs apply_regression + apply_regression_bandpass on the warped BOLD. No regressor recomputation -- same matrix, different target space. bold_mask is now mandatory (was Path | None). BIDS export writes per-regressor desc-regressed and desc-preproc BOLD with reg-<strategy> entity. resolve_longitudinal_func resolves the raw regressor files per strategy. CLI gains --regressor with the same choices as cross-sectional (36-parameter, aCompCor). Port of the longitudinal regression logic from #282. Co-authored-by: Janhavi Pillai <janhavi.pillai@gmail.com> * Add tests for longitudinal functional regression (Stage 5 of #301) Unit tests: resolve_longitudinal_func with single/multiple regressors, missing regressor raises, mandatory bold_mask; export file counts and reg-<strategy> entity validation. Tier-2 integration: apply_regression and apply_regression_bandpass on warped BOLD with raw regressors produce non-degenerate outputs. Tier-4 full_pipeline: longitudinal functional outputs exist, regressed and cleaned BOLD have non-zero variance, bold mask is binary. * Remove redundant regressor_set, replace fake tests with real ds000114 integration - Drop regressor_set param from longitudinal_process; iterate regressor_files.keys() instead. The resolve layer already filters to the requested strategies, so the dict keys ARE the set. Eliminates the caller sync burden. - Delete tests/full_pipeline/longitudinal/ -- the conftest hacked the cross-sectional anat brain as a "longitudinal template", which exercises the code path but not the actual transform chain. - Rewrite tests/integration/longitudinal/test_regression_reuse.py as a proper end-to-end CLI test: rbc functional -> rbc longitudinal functional on ds000114 sub-01 ses-test via subprocess. Asserts expected BIDS tree, non-degenerate variance, and binary mask. - Add ds000114_func_derivatives and longitudinal_func_output fixtures to the integration conftest so the full cross-sectional -> longitudinal chain runs on real multi-session data with docker. * Fix integration test fixture ordering to avoid bids2table discovery issue ds000114_func_derivatives now depends on ds000114_anat_derivatives directly instead of longitudinal_template_output. This ensures rbc functional runs before the template step writes ses-longitudinal files into the derivatives dir, avoiding a potential bids2table discovery issue where the ses-longitudinal anat files interfere with the functional pipeline's anat resolution. longitudinal_func_output depends on both ds000114_func_derivatives and longitudinal_template_output to ensure both cross-sectional functional outputs and the longitudinal template are present before rbc longitudinal functional runs. * Debug: log derivatives tree before rbc functional in integration fixture * Debug: dump bids2table output for derivatives dir before rbc functional * Debug: inline rbc functional call with verbose + tree dump on failure * Fix integration fixture: --task filter drops anat rows Filters.apply() applies the --task filter to ALL rows including anat. Anat files have task=null, so --task fingerfootlips drops them, causing resolve_functional to fail with FileNotFoundError on desc-brain_T1w. Drop --task from both rbc functional and rbc longitudinal functional fixture calls. ds000114 only has one task per session so the filter isn't needed. Remove debug logging from previous commits. * Fix Filters.apply() task filter dropping anat rows The --task filter applied pl.col("task") == value to all rows, but anat/dwi files have task=null, so they were silently dropped. This caused rbc functional --task <label> to fail with FileNotFoundError on desc-brain_T1w for any dataset. Fix: use pl.col("task").is_null() | (pl.col("task") == value) so rows without a task entity (anat, dwi, fmap) pass through. Affects functional and all orchestration; metrics/qc pre-filter to datatype=func so they were never hit. Restore --task in integration fixtures now that the fix is in place. * Fix expected filenames: reg entity precedes desc in BIDS builder output --------- Co-authored-by: Janhavi Pillai <janhavi.pillai@gmail.com>

jpillai00 marked this pull request as ready for review April 8, 2026 23:44

jpillai00 requested review from Copilot, kaitj and nx10 April 8, 2026 23:44

Copilot started reviewing on behalf of jpillai00 April 8, 2026 23:45 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

Comment thread src/rbc/orchestration/longitudinal.py Outdated

Comment thread src/rbc/bids/longitudinal.py

kaitj reviewed Apr 9, 2026

View reviewed changes

Comment thread src/rbc/cli/longitudinal.py Outdated

jpillai00 force-pushed the add/long-func-outputs branch from ce86168 to 657623e Compare April 9, 2026 15:13

jpillai00 linked an issue Apr 9, 2026 that may be closed by this pull request

Update longitudinal functional workflow outputs #190

Closed

jpillai00 added 5 commits April 10, 2026 12:59

longitudinal func outputs

d5bf9c7

rebase and tests

30d6d5f

inputs and tests

beecfe3

bpf regressors naming

45babbc

changes

d4fa651

jpillai00 force-pushed the add/long-func-outputs branch from 657623e to d4fa651 Compare April 10, 2026 20:08

fix args

90cd371

jpillai00 mentioned this pull request Apr 10, 2026

Longitudinal metrics workflow Fix #200 #297

Closed

nx10 mentioned this pull request Apr 13, 2026

Longitudinal pipeline: refactor and release readiness (tracker) #301

Closed

8 tasks

nx10 closed this Apr 13, 2026

nx10 mentioned this pull request Apr 16, 2026

Functional outputs & regressor reuse (Stage 5 of #301) #315

Merged

7 tasks

kaitj deleted the add/long-func-outputs branch May 20, 2026 23:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Additional outputs to longitudinal functional workflow Fix #190#282

Additional outputs to longitudinal functional workflow Fix #190#282
jpillai00 wants to merge 6 commits into
mainfrom
add/long-func-outputs

jpillai00 commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

kaitj left a comment •

edited

Loading

Uh oh!

Uh oh!

nx10 commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jpillai00 commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

kaitj left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nx10 commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Apr 8, 2026 •

edited

Loading

kaitj left a comment •

edited

Loading